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Novel  Vision  Sensors  that  Approximate  Wide-Angle  Perspective  Projections: 

Reference  [2]  by  Hicks  and  Bajcsy,  present  two  families  of  reflective  surfaces  that  are  cabable  of 
providing  a  wide  field  of  view,  and  yet  still  approximate  a  perspective  projection  to  a  high  degree. 
These  surfaces  are  derived  by  considering  a  plane  perpendicular  to  the  axis  of  the  surface  of  revo¬ 
lution  and  finding  the  equations  governing  the  distortion  of  the  image  of  the  plane  in  this  surface. 
We  then  view  this  relation  as  a  differential  equation  and  prescribe  the  distortion  term  to  be  linear. 
By  choosing  appropriate  initial  conditions  for  the  differential  equation  and  solving  it  numerically, 
we  derive  the  surface  shape  and  obtain  a  precise  estimate  as  to  what  degree  the  the  resulting  sensor 


1 


can  approximate  a  perspective  projection.  Thus,  these  surfaces  act  as  computational  sensors,  al¬ 
lowing  for  a  wide-angle  perspective  view  of  a  scene  without  processing  the  image  in  software.  The 
applications  of  such  a  sensor  should  be  numerous,  including  surveillance,  robotics,  and  traditional 
photography. 
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When  and  Where  Will  AI 
Meet  Robotics? 

Issues  in  Representation 

Ruzena  Bajscy  and  Edward  W.  Large 


■  Because  perception-action  systems  are  necessarily 
constrained  by  the  physics  of  time  and  space, 
robotodsts  often  assume  they  are  best  described 
using  differential  equations,  a  language  that  is  spe- 
dalized  for  describing  the  evolution  of  variables 
that  represent  physical  quantities.  However,  when 
it  comes  to  decision  making,  where  the  represen¬ 
tations  involved  refer  to  goals,  strategies,  and  pref¬ 
erences,  Al  offers  a  diverse  range  of  formalisms  to 
the  modeler.  However,  the  relationship  between 
these  two  levels  of  representation — signal  and 
symbol — are  not  well  understood.  If  we  are  to 
achieve  success  in  modeling  intelligent  physical 
agents,  robotics  and  AI  must  reach  a  new  consen¬ 
sus  on  how  to  integrate  perception-action  systems 
with  systems  designed  for  abstract  reasoning. 

In  the  early  days  of  AI,  robotics  was  an  inte¬ 
gral  part  of  our  research  effort.  All  our  major 
AI  laboratories  had  research  programs  in 
robotics  in  the  late  1960s  and  early  1970s. 
However  by  the  1980s,  robotics  had  taken  its 
own  course  separate  from  the  core  activities  of 
AI.  One  might  argue  that  such  a  split  was 
inevitable,  a  natural  result  of  spedalization  in 
a  rapidly  growing  and  maturing  field  such  as 
ours,  but  in  our  pursuit  of  rational  models  of 
the  mind,  do  we  dare  leave  the  body  behind? 

What  is  responsible  for  the  divergence 
between  these  two  fields  that  once  were  so  inti¬ 
mately  intertwined?  Can  AI  and  robotics  ever 
be  reunited,  and  if  so,  what  would  a  new  part¬ 
nership  look  like?  At  the  core,  we  believe,  is  the 
ubiquitous  issue  of  representation.  There  is  an 
enormous  difference  between  dealing  with 
physical  systems  that  operate  in  our  everyday 
environment  and  software  systems  that  reside 
in  various  abstract  worlds.  This  gap  has  led  to 
divergence  in  many  areas,  including  the  fol¬ 
lowing: 

The  problems:  Robotics  problems  entail  sys¬ 


tems  and  agents  interacting  with  the  physical 
world,  but  AI  deals  mostly  with  abstract  prob¬ 
lems  that  lend  themselves  to  symbolic  repre¬ 
sentations. 

The  environment:  Roboticists  seek  to 
design  systems  that  function  in  physical  envi¬ 
ronments  that  are  always  changing  and  intrin¬ 
sically  unpredictable.  Software  agents  generally 
operate  in  human-designed  worlds  where  one 
can  have  some  measure  of  control  over  change 
or  at  least  an  a  priori  knowledge  of  the  possibil¬ 
ities. 

The  tools:  AI  more  commonly  uses  discrete 
mathematics,  but  robotics  and  machine  per¬ 
ception  make  use  of  continuous  mathematics. 
These  tools  also  differentiate  the  typical  educa¬ 
tional  background  that  characterizes  the  two 
areas:  AI  has  more  computer  scientists,  but 
robotics  has  more  electrical  and  mechanical 
engineers. 

The  evaluation  criteria:  AI  researchers  seem 
to  value  novelty  more,  solving  "hard"  prob¬ 
lems,  showing  existence  of  solutions,  and  so 
forth.  Robotics,  however,  follows  traditional 
engineering  evaluation  criteria:  efficiency,  reli¬ 
ability,  accuracy  of  performance,  and  economy 
of  the  solution. 

We  admit  that  these  divisions  might  be 
overexaggerated  because  in  both  fields,  one 
can  find  counterexamples  to  the  previous 
statements.  Nevertheless,  each  of  these  points 
speaks  to  the  differences  one  encounters  when 
dealing  with  corporeal  agents  in  the  physical 
world  versus  software  agents  in  cyberspace. 
Robotics  concentrates  most  of  its  resources  on 
modeling  perception  and  action.  Often,  differ¬ 
ential  equations  are  used  to  embody  relatively 
simple  strategies  for  controlling  hardware 
effectors  based  on  sensory  information.  AI, 
however,  emphasizes  planning  and  abstract 
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Figure  1.  The  Autonomous  Agent 

The  TRC  platform  serves  as  a  mobile  base.  A  stereo  camera  pair  mount¬ 
ed  on  the  front  of  the  rig  is  used  for  obstacle  detection.  A  third  camera 
mounted  on  a  pan  platform  is  used  for  target  detection  and  tracking. 


reasoning.  For  example,  logical,  grammatical, 
or  other  discrete  formalisms  are  used  to  model 
the  complex  operations  involved  in  winning  a 
chess  match  or  parsing  a  sentence. 

What  seems  clear  is  that  as  robotic  agents 
are  called  on  to  perform  increasingly  complex 
tasks,  they  will  be  required  not  only  to  react 
flexibly  in  dynamically  changing  environ¬ 
ments  but  also  to  make  decisions,  reason 
abstractly,  and  change  perceptual  or  behav¬ 
ioral  strategies.  Conversely,  as  intelligent  soft¬ 
ware  agents  are  required  to  operate  more  and 
more  on  human  terms,  responding  to  sensory 


information  and  interacting  physically  with 
humans,  they  will  be  required  to  integrate 
more  sophisticated  perception-action  capabili¬ 
ties  with  their  abstract  reasoning  abilities.  In 
addition,  although  each  of  these  areas  has 
been  well  studied,  in  robotics  and  AI  respec¬ 
tively,  the  integration  of  perception-action  sys¬ 
tems  with  reasoning  systems  is  less  well  under¬ 
stood.  Thus,  there  is  a  great  need  to  reconsider 
the  relationship  between  AI  and  robotics. 


The  Problem  of  Representation 

We  will  explore  the  issue  that  we  believe  is  key 
to  this  relationship,  the  issue  of  representa¬ 
tion.  Representation  is  critical,  especially  when 
one  considers  how  to  find  a  description  that  is 
compact,  yet  expressive  enough  to  enable  the 
modeling  of  intelligent  physical  agents.  What 
kind  of  mathematical  tools  are  available  to  us? 
At  the  signal  level,  modeling  of  perception- 
action  (reactive)  behaviors  can  be  anchored  in 
differential  equations  and  control  theory,  both 
linear  and  nonlinear.  At  the  symbol  level, 
higher-level  control  derives  its  models  either 
from  geometry  (typically  used  in  robotics)  or 
from  logics  and  rule-based  systems  such  as  are 
favored  in  the  AI  planning  community.  If  time 
needs  to  be  explicitly  accounted  for,  then  there 
are  other  tools  available.  At  the  signal  level, 
time  is  implicit  in  the  model  of  the  reactive 
behaviors  (for  example,  using  differential 
equations).  At  the  symbol  level,  discrete  states 
are  generally  considered,  and  these  can  be 
modeled  using  discrete-event  systems;  tempo¬ 
ral  logics;  and,  at  an  even  higher  level,  fluents. 
If  uncertainty  and  disturbances  must  be  mod¬ 
eled,  then  one  must  bring  to  bear  stochastic 
models  and  probability  theory;  partially 
observable  Markov  decision  models  is  one 
such  example.  Finally,  utility  functions  and 
cost-benefit  trade-offs  come  to  play  in  con¬ 
junction  with  game  theory,  optimization, 
selection  of  strategies,  and  complexity  consid¬ 
erations.  Examples  of  such  approaches  as  they 
have  been  applied  to  robotics  are  presented  in 
Alami  et.  al.  (1998);  Clementia,  Di  Felice,  and 
Hernandez  (1997);  Cohn  (1995);  Russell  and 
Subramanian  (1995);  and  Sandewall  (1994). 
Hybrid  system  approaches  such  as  discussed  in 
Arkin  (1998),  Brockett  (1993),  Dickmanns 
(1997),  Nagel  and  Haag  (1998),  Nerode  and 
Remmel  (1991),  and  Ramadge  and  Wonham 
(1987)  combine  discrete  systems  with  lower- 
level  control  systems.  However,  the  use  of  all 
these  mathematical  tools  is  predicated  on  the 
assumption  that  label  assignment  (the  seg¬ 
mentation  of  the  sensory  or  control  signals)  is 
performed  externally  to  the  system. 
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Discrete-Event  System  Competitive  Dynamic  System 


Figure  2.  Two  Approaches  to  Decision  Making  and  Behavioral  Sequencing. 

In  the  discrete-event  system,  the  discrete  states  of  a  finite-state  machine  correspond  to  control  of  perception-action  by  a  dis¬ 
tinct  control  law  (0= Fstate  (0, 40).  Transitions  between  states  are  governed  by  guard  conditions  on  perceptual  variables  (for 
example,  y/).  In  the  competitive  dynamic  system,  each  state  variable  (for  example,  w)  contrpls  the  weighting  of  a  task  con¬ 
straint  at  the  signal  level.  Behavior  is  shaped  directly  by  the  competitive  dynamic  system,  0  =  F(0,  W,  40,  as  the  symbol- 
level  system  activates  and  deactivates  attractor  and  repellor  contributions  to  the  behavioral  dynamics.  As  perceptual  parame¬ 
ters  (for  example,  y)  change,  bifurcations  cause  qualitative  changes  in  perception-action  behavior. 


Signal  and  Symbol 

In  our  view,  the  main  unsolved  problem  is 
how  to  segment  the  continuous  signal  into 
states,  strategies,  labels,  and  so  forth,  and  how 
to  arbitrate  among  states  for  effective  interac¬ 
tion  with  the  environment  (for  example,  Shi 
and  Malik  [1998],  Tari,  Shah,  and  Pien  [1997], 
and  Large,  Christensen,  and  Bajscy  [1999]).  In 
the  GRASP  Laboratory,  we  have  concerned 
ourselves  with  the  problem  of  representation 
in  signal-symbol  systems  over  the  past  several 
years.  One  approach  to  this  problem  involves 
the  study  of  intelligent  physical  agents ,  agents 
that  can  operate  in  the  physical  world  with  all 
its  uncertainty  yet  behave  intelligently,  mak¬ 
ing  decisions  about  how  best  to  perform  sim¬ 
ple  and  complex  tasks  in  a  range  of  real-world 
environments.  A  picture  of  one  such  agent  is 
shown  in  figure  1.  It  consists  of  a  TRC  labmate 


mobile  platform  equipped  with  a  stereo  cam¬ 
era  pair  used  for  obstacle  detection;  a  third 
camera  mounted  on  a  turntable  for  visual 
tracking;  and  several  computers  used  for  pro¬ 
cessing  sensory  signals,  generating  control  sig¬ 
nals,  and  making  decisions. 

The  control  system  that  models  perception- 
action  behavior  transforms  visual  input  into 
control  signals  to  enable  the  physical  agent  to 
carry  out  various  navigation  tasks.  We  use  a 
dynamic  system  approach  proposed  by  Schon- 
er,  Dose,  and  Engels  (1996).  In  this  approach, 
behavior  is  controlled  by  differential  equations 
that  describe  the  rate  of  change  of  behavioral 
variables,  such  as  heading  direction  and  veloc¬ 
ity.  At  any  instant  in  time,  the  values  of  these 
variables  describe  the  agent's  behavior.  Over 
time,  the  dynamic  system  generates  a  series  of 
values,  controlling  the  behavior  of  the  agent. 
Our  dynamic  system  has  the  form 
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Although 
the  dynamic 
control  system 
provides  a 
great  deal  of 
flexibility,  it 
can  only 
model  one 
relatively 
simple 
perception- 
action 
behavior 
at  a  time. 
Complex 
tasks, 
however, 
typically 
require  the 
execution  of 
sequences  of 
behavior. 


D®/dt  =  F(@,V )  (1) 

where  0  =  [4>  v]r  is  a  vector  of  behavioral  vari¬ 
ables;  heading  direction;  and  velocity  and  is  a 
vector  of  variables  that  represent  perceptual 
information;  such  as  direction  to  the  target 
and  size  of  obstacles.  An  example  of  such  a 
function  is  shown  in  figure  2  (bottom  panel). 
Three  fixed  points  can  be  seen  in  the  figure  as 
points  where  the  value  of  ft0,1If)  is  zero  (that 
iS;  dfy/dt  =  0;  so  heading  direction  is  fixed).  If 
the  slope  of  the  function  around  a  fixed  point 
is  positive,  the  value  of  the  behavioral  variable 
is  pushed  away  from  this  value  by  the  action  of 
equation  1;  such  an  unstable  fixed  point  is 
called  a  repellor .  If  the  slope  of  the  function  is 
negative,  it  is  a  stable  fixed  point,  called  an 
attractor,  because  the  behavioral  variable  is 
pulled  toward  this  value  by  the  differential 
equation. 

The  behavior  of  the  agent  is  controlled  by 
the  configuration  of  attractors  and  repellors: 
Desired  actions  (such  as  moving  toward  a  tar¬ 
get)  are  modeled  as  attractors,  and  undesired 
actions  (such  as  moving  toward  an  obstacle) 
are  modeled  as  repellors  of  the  perception- 
action  dynamic  system.  Task  constraints  deter¬ 
mine  the  mapping  from  perceptual  informa¬ 
tion  to  behavioral  attractors  and  repellors.  If 
the  task  is  to  go  to  the  desk,  the  action  of  mov¬ 
ing  toward  the  desk  is  modeled  an  attractor, 
but  other  objects  are  considered  obstacles 
(modeled  as  repellors).  However,  if  the  task  is 
to  rendezvous  with  another  agent,  then  the 
action  of  moving  toward  the  other  agent  is  an 
attractor,  and  the  desk  is  treated  as  an  obstacle, 
and  moving  toward  it  is  to  be  avoided.  Thus, 
viewed  as  a  representation  of  a  perception- 
action  behavior,  this  dynamic  system  incorpo¬ 
rates  task  knowledge  and  makes  use  of  percep¬ 
tual  information. 

As  the  values  of  the  perceptual  variables 
change,  the  attractor-repellor  layout  changes. 
For  example,  if  a  target  moves,  the  location  of 
the  corresponding  attractor  will  move  as  well, 
thus  providing  behavioral  flexibility— behav¬ 
ior  adapts  to  accommodate  changes  in  the 
environment.  An  even  greater  measure  of  flex¬ 
ibility  is  provided  by  bifurcations  in  the  dynam¬ 
ic  system — qualitative  changes  in  the  layout  of 
attractors  and  repellors  caused  by  changes  in 
parameter  values.  For  example,  when  a  new 
obstacle  comes  into  view,  a  repellor  forms 
where  there  was  no  repellor  before. 

Although  the  dynamic  control  system  pro¬ 
vides  a  great  deal  of  flexibility,  it  can  only 
model  one  relatively  simple  perception-action 
behavior  at  a  time.  Complex  tasks,  however, 
typically  require  the  execution  of  sequences  of 
behavior.  For  example,  one  agent  might  need 


to  rendezvous  with  an  agent  and  then  proceed 
toward  a  target  location.  To  perform  such  tasks 
in  a  complex  environment  requires  sequenc¬ 
ing  several  simpler  perception-action  strate¬ 
gies.  Building  on  the  previous  signal-level  rep¬ 
resentation,  we  have  investigated  two 
approaches  to  modeling  decision  making  and 
sequence  generation:  (1)  a  discrete-event-system 
approach  and  (2)  a  competitive  dynamic  system 
approach.  There  are  three  key  differences 
between  these  two  approaches:  (1)  the  model 
of  how  the  symbol  level  interfaces  with  the  sig¬ 
nal  level,  (2)  the  model  of  how  perception  is 
integrated  into  the  decision-making  process, 
and  (3)  the  model  of  how  decisions  are  cap¬ 
tured  at  the  symbol  level. 

Before  describing  each  approach  in  detail, 
we  summarize  the  differences  between  the  two 
systems  in  figure  2.  In  the  discrete-event -  system 
approach  (Kosecka  1996),  the  symbol  level 
interfaces  with  the  signal  level  by  realizing  dis¬ 
tinct  behaviors  as  separate  dynamic  systems 
(that  is,  fstate,  middle  left  panel);  these  are  con¬ 
ceived  of  as  elementary  perception-action 
strategies.  At  any  particular  time,  the  behavior 
of  the  agent  is  governed  by  one  of  these  equa¬ 
tions.  Decision  making  and  sequencing  are 
modeled  at  the  symbol  level  using  a  finite-state 
machine  (FSM)  (top  left  panel).  Individual  FSM 
states  correspond  to  elementary  signal-level 
behaviors;  when  in  a  particular  state,  behavior 
is  governed  by  a  corresponding  signal-level 
dynamic  system.  The  arcs  linking  states  are 
labeled  with  discrete  events  (for  example,  a  > 
0)  that  summarize  perceptual  information.  The 
perceptual  system  generates  these  events, 
which  correspond  to  qualitatively  different 
conditions  in  the  environment.  The  occur¬ 
rence  of  a  specific  event  causes  the  switch  from 
one  state  to  another,  modeling  the  decision  to 
execute  a  different  perception-action  behavior. 
Thus,  sequences  of  behavior  are  generated  by 
traversing  the  arcs,  which  is,  in  turn,  governed 
by  the  conditions  on  the  current  situation. 

The  competitive  dynamic  system  model  is  for¬ 
mulated  entirely  within  the  qualitative  theory 
of  dynamic  systems.  Both  signal-level  control 
and  symbol-level  decision  making  are  modeled 
using  differential  equations  (Large,  Chris¬ 
tensen,  and  Bajscy  1999).  However,  the  com¬ 
petitive  dynamic  system  interfaces  with  the  sig¬ 
nal  level  differently  than  the  discrete-event 
system.  Rather  than  defining  multiple  elemen¬ 
tary  perception-action  behaviors,  a  single  mas¬ 
ter  equation  is  defined  containing  all  possible 
task  constraints  (middle  right  panel,  figure  2). 
Then,  each  variable  in  the  symbol-level  system 
controls  the  weighting  of  a  task  constraint  at 
the  signal  level,  such  that  the  symbol-level  sys- 
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Figure  3.  Finite-State  Machine  (FSM)  Models  for  Simple  and  Complex  Behaviors . 

A.  An  FSM  for  elementary  behavior  GoTo.  The  control  law  (fGoTo)  is  repeatedly  invoked  in  the  next  state  until  suc¬ 
cessful  (arrival  at  the  goal)  or  unsuccessful  (for  example,  detection  of  a  spurious  attractor)  termination. 

B.  Finite-state  model  for  a  navigation  behavior.  Failure  of  GoTo  is  followed  by  the  elementary  behavior  Escape.  Once 
the  agent  clears  the  obstruction,  GoTo  is  invoked  again.  This  more  complex  behavior  is  able  to  handle  a  large  variety 
of  navigation  situations. 

tem  can  activate  and  deactivate  attractor  and  teresis,  govern  decision  making  and  sequence 
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Figure  4.  Bifurcations  in  a  Competitive  Dynamic  System  for  Decision  Making. 

Two  bifurcation  diagrams  are  shown,  one  corresponding  to  each  dimension  of  a  two-dimensional  system  for  system  navigation.  It  is 
assumed  that  Ytar/obs =  Yobstar”  °-5*  A.  The  state  variable  wobs  determines  the  weighting  of  ftar.  B.  The  state  variable  wobs  determines  the 
weighting  of  foh!.  Four  qualitatively  different  behaviors,  corresponding  to  four  fixed  points  of  the  competitive  dynamics,  are  shown. 
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models  of  more  complex  tasks,  which  are 
described  as  sequences  of  behavioral  atoms.  By 
using  a  task-specification  language,  complex 
FSMs  are  synthesized  by  sequencing  simpler 
automata. 

The  FSM  model  of  an  elementary  GoTo 
strategy  is  shown  in  figure  3a.  A  signal-level 
perception-action  behavior  is  repeatedly 
invoked  in  the  next  state  until  the  agent  reach¬ 
es  the  desired  target  location  and  makes  a  tran¬ 
sition  to  the  final  state.  If  the  strategy  fails,  the 
transition  to  the  unsuccessful  final  state  is 
made.  As  an  example  of  a  composition  of  ele¬ 
mentary  behaviors,  consider  the  problem  of 
moving  to  a  target  location  while  avoiding 
obstacles.  In  simple  environments,  one  ele¬ 
mentary  perception-action  behavior,  GoTo, 
might  do  the  trick.  However,  in  complex  envi¬ 
ronments  with  multiple  obstacles  arrayed  in 
difficult  configurations,  our  agent  might  get 
stuck  in  an  area  and  never  reach  the  target 
location  (Large,  Christensen,  and  Bajscy  1999). 
We  address  this  problem  by  adding  an  Escape 
behavior  that  enables  the  agent  to  find  its  way 
out  of  enclosures  and  other  spatial  traps.  The 
FSM  for  this  behavior  is  shown  in  figure  3b.  We 
assume  that  the  fail  signal  to  the  GoTo  behav¬ 
ior  is  generated  whenever  the  agent  detects  an 
enclosure  from  which  it  must  escape.  When 
the  fail  signal  is  generated,  the  FSM  enters  the 
unsuccessful  final  state,  and  a  transition  is 
made  to  the  initial  state  of  Escape.  When 
Escape  terminates  (when  the  agent  has  cleared 
the  obstruction),  the  GoTo  behavior  resumes. 
The  navigation  task  terminates  successfully 
when  the  agent  reaches  the  target  location. 

Competitive  Dynamic  Systems 

The  competitive  dynamic  system  strategy 
models  individual  behaviors  as  the  stable  fixed 
points  of  a  decision-making  dynamic  system. 
This  system  interacts  with  the  signal  level  not 
by  invoking  separate  elementary  behaviors  but 
by  directly  shaping  perception-action  strate¬ 
gies.  The  variables  of  the  competitive  dynamic 
system  determine  the  weighting  of  the  task 
constraints  in  the  behavioral  dynamic  system. 
This  interface  between  the  two  levels  allows 
the  decision-making  system  to  activate  and 
deactivate  attractors  and  repellors  in  the  per¬ 
ception-action  system,  synthesizing  control 
laws  on  the  fly.  Thus,  qualitatively  different 
configurations  of  the  weights  give  rise  to  dis¬ 
tinct  perception-action  behaviors.  In  addition, 
distinct  weight  configurations,  which  arise  as 
attractors  in  the  competitive  dynamic  system, 
are  functionally  equivalent  to  the  FSM  states  of 
the  discrete-event  system. 

Decisions  are  made  through  bifurcations  in 


the  competitive  dynamic  system,  and  as  with 
any  dynamic  system,  bifurcations  are  caused 
by  changes  in  the  values  of  the  system  parame¬ 
ters.  The  decision-making  system  uses  two 
types  of  parameter:  (1)  competitive  advantage 
and  (2)  competitive  interaction.  These  parame¬ 
ters  are  tied  to  perceptual  information,  so  that 
decisions  are  made  on  the  basis  of  the  environ¬ 
ment  as  sensed  by  the  agent.  First,  each  weight 
has  an  associated  competitive  advantage  that 
describes  whether  the  corresponding  task  con¬ 
straint  (for  example,  move  toward  target, 
avoid  obstacles)  is  appropriate  to  the  agent's 
current  situation.  For  example,  if  obstacles  are 
nearby,  the  Obstacles  constraint  will  have  a 
strong  competitive  advantage,  but  if  the  target 
is  also  in  view,  the  Target  constraint  also  has  a 
strong  advantage.  The  activation  of  both  con¬ 
straints  simultaneously  corresponds  to  the  ele¬ 
mentary  GoTo  behavior  of  the  discrete-event 
system  earlier.  Competitive  interaction  describes 
the  extent  to  which  each  constraint  is  consis¬ 
tent  or  inconsistent  other  constraints.  For 
example,  if  the  agent  finds  itself  enclosed  in  an 
area  with  the  target  just  beyond,  the  competi¬ 
tive  interaction  between  the  Obstacles  and  the 
Target  constraints  would  increase  so  that  the 
Target  consttaint  would  be  deactivated  tem¬ 
porarily,  allowing  the  agent  to  escape  from  the 
enclosure.  Deactivation  of  the  target  con¬ 
straint  corresponds  to  the  Escape  behavior  of 
the  discrete-event  system. 

We  can  understand  in  detail  how  these  para¬ 
meters  interact  to  determine  the  behavior  of 
the  agent  by  constructing  a  bifurcation  dia¬ 
gram  such  as  that  in  figure  4.  The  bifurcation 
diagram  shows  the  layout  of  fixed  points  for  a 
two-dimensional  system  (that  is,  weights  for 
the  Target  and  Obstacles  constraints)  as  a  func¬ 
tion  of  the  perceptual  parameters.  In  the  fig¬ 
ure,  the  competitive  advantage  parameters  are 
varied  from  0  to  1,  assuming  that  the  two  com¬ 
petitive  interaction  parameters  remain  fixed  at 
0.5.  Four  qualitatively  different  regions  (and, 
thus,  behaviors)  are  pictured.  Activation  of 
both  constraints  corresponds  to  the  GoTo 
behavior  described  earlier,  but  activation  of 
the  Obstacles  constraint  only  corresponds  to 
the  Escape  behavior. 

Beginning  in  the  front  left  corner  of  the 
parameter  space,  only  the  Obstacles  constraint 
contributes  to  the  behavioral  dynamics  (the 
Escape  behavior).  Moving  to  the  right,  as  Tar¬ 
get's  advantage  increases  beyond  0.5,  both  Tar¬ 
get  and  Obstacles  constraints  contribute  to 
shape  perception-action  behavior  (the  GoTo 
behavior).  As  we  next  decrease  the  advantage 
of  Obstacles,  moving  to  the  back  left  region,  x 
Obstacles  is  deactivated,  but  Target  is  activat- 
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ed.  If  we  once  again  decrease  Target,  moving 
toward  the  back  left  region  of  the  parameter 
space,  we  notice  that  Obstacles  remains  inac¬ 
tive,  but  Target  remains  active.  This  last  region 
of  the  state  space  is  different  from  the  others. 
Two  behaviors  are  stable  in  this  region.  How¬ 
ever,  the  system  can  only  occupy  one  of  these 
states  at  any  given  time.  In  this  case,  the  state 
of  the  system  is  determined  by  its  recent  histo¬ 
ry,  a  phenomenon  known  as  hysteresis.  Finally, 
the  boundaries  of  the  four  regions  are  deter¬ 
mined  by  the  values  of  the  competitive  inter¬ 
action  parameters.  When  competitive  interac¬ 
tions  change,  the  relative  sizes  of  the  different 
stable  regions  change  as  well.  Each  of  these  dif¬ 
ferent  regions  corresponds  to  the  execution  of 
a  qualitatively  different  perception-action 
behavior. 

Comparing  Approaches 
We  tested  the  two  systems  described  previous¬ 
ly  to  evaluate  their  relative  performance  in 
autonomous  navigation  tasks  (Large,  Chris¬ 
tensen,  and  Bajscy  1999).  We  vary  environ¬ 
mental  complexity  by  constructing  environ¬ 
ments  with  different  numbers  of  obstacles 
arranged  in  various  configurations.  We  also 
vary  task  complexity;  for  example,  a  single 
agent  performs  simple  navigation,  or  a  pair  of 
agents  performs  a  cooperative  task.  In  a  range 
of  tests,  dynamic  agents  perform  tasks  faster 
and  more  reliably  than  discrete-event  agents. 
They  are  able  to  maintain  higher  mean  veloci¬ 
ties,  finding  targets  faster  and  with  lower  fail¬ 
ure  rates  than  discrete-event  agents.  However, 
the  discrete-event  agents  also  have  advantages. 
They  obey  task  constraints  more  faithfully, 
reluctant  to  relax  constraints  regardless  of 
environmental  complexity.  These  differences 
are  the  result  of  the  model  of  decision  making. 
The  symbol-level  dynamic  system  changes 
state  less  often  than  the  discrete-event  system, 
especially  in  the  face  of  a  noisy  sensor  reading. 
Although  our  experiments  are  not  yet  conclu¬ 
sive,  by  comparing  modeling  strategies  in  a 
careful  way,  we  are  gaining  important  insights 
into  the  special  requirements  of  systems  that 
must  manipulate  both  signals  and  symbols  at 
the  same  time  and  toward  the  same  goal. 

Conclusions 

Early  in  the  history  of  AI,  many  researchers 
came  to  believe  that  perception  and  action 
could  be  modeled  by  relatively  simple  trans¬ 
duction  mechanisms,  and  therefore,  abstract 
reasoning  and  problem  solving  were  the  diffi¬ 
cult  issues  worthy  of  study.  More  recently,  it 
has  been  argued  that  complex  representation 


and  reasoning  might  be  unnecessary  because 
many  apparently  intelligent  behaviors  can  be 
modeled  as  perception-action  systems  situated 
in  the  physical  world.  Unfortunately,  we  have 
come  to  view  both  points  of  view  as  somewhat 
simplistic.  Intelligent  agents  must  be  capable 
of  bringing  to  bear  a  rich  variety  of  perception- 
action  strategies  but,  at  the  same  time,  reason¬ 
ing  and  solving  problems  to  perform  both 
familiar  and  unfamiliar  tasks  in  novel  environ¬ 
ments. 

In  this  regard,  the  study  of  intelligent  phys¬ 
ical  agents  and  their  behavior  is  of  tremendous 
theoretical  and  practical  significance  in  AI.  Not 
only  are  there  a  vast  number  of  real-world 
applications  where  autonomous  agents  can  be 
useful,  but  models  of  intelligent  physical 
agents  can  serve  as  valuable  starting  points  for 
theories  of  intelligent  biological  systems.  The 
question  that  we  ask  is  how  to  integrate  mod¬ 
els  of  perception-action  behavior  with  models 
of  problem-solving  behavior. 

Although  we  do  not  yet  have  the  answer,  we 
have  two  requirements  for  any  solution:  First, 
the  description  of  the  physical  agent  should 
take  place  within  a  structured  framework  that 
supports  both  analysis  and  theory  making. 
Thus,  we  are  allowed  to  develop  design 
methodologies  for  artificial  agents  as  well  as 
develop  rational  theories  of  biological  agents. 
Furthermore,  any  methodology  that  we  pro¬ 
pose  should  be  compositional,  allowing  man¬ 
ageable  and  flexible  system  design  through 
decomposition  of  complex  problems  or  behav¬ 
iors  into  subparts.  Both  of  our  systems  meet 
these  requirements.  Finally,  it  is  necessary  to 
carefully  compare  the  assumptions  brought  to 
bear  by  different  strategies  as  we  learn  to  mod¬ 
el  intelligent  behavior  in  the  real  world. 

What  are  the  special  requirements  of  sys¬ 
tems  that  must  interact  with  the  physical 
world  and  also  reason  and  solve  problems?  It  is 
this  question  that  must  be  addressed  before  we 
can  claim  a  theory  of  intelligent  physical 
agents  and  before  AI  and  robotics  can  be 
reunited. 
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Abstract 

We  present  two  families  of  reflective  surfaces  that  are 
capable  of  providing  a  wide  field  of  view,  and  yet  still  ap¬ 
proximate  a  perspective  projection  to  a  high  degree.  These 
surfaces  are  derived  by  considering  a  plane  perpendicular  to 
the  axis  of  a  surface  of  revolution  and  finding  the  equations 
governing  the  distortion  of  the  image  of  the  plane  in  this  sur¬ 
face.  We  then  view  this  relation  as  a  differential  equation 
and  prescribe  the  distortion  term  to  be  linear.  By  choos¬ 
ing  appropriate  initial  conditions  for  the  differential  equa¬ 
tion  and  solving  it  numerically,  we  derive  the  surface  shape 
and  obtain  a  precise  estimate  as  to  what  degree  the  result¬ 
ing  sensor  can  approximate  a  perspective  projection.  Thus 
these  surfaces  act  as  computational  sensors,  allowing  for  a 
wide-angle  perspective  view  of  a  scene  without  processing 
the  image  in  software.  The  applications  of  such  a  sensor 
should  be  numerous,  including  surveillance,  robotics  and 
traditional  photography. 

Recently,  many  researchers  in  the  robotics  and  vision 
community  have  begun  to  consider  visual  sensors  that  are 
able  to  obtain  wide  fields  of  view.  Such  devices  are  the  nat¬ 
ural  solution  to  various  difficulties  encountered  with  con¬ 
ventional  imaging  systems. 

The  two  most  common  means  of  obtaining  wide  fields  of 
view  are  fish-eye  lenses  and  reflective  surfaces,  also  known 
as  catoptrics.  When  catoptrics  are  combined  with  conven¬ 
tional  lens  systems,  known  as  dioptrics,  the  resulting  sen¬ 
sors  are  known  as  catadioptrics.  The  possible  uses  of 
these  systems  include  applications  such  as  robot  control  and 
surveillance.  In  this  paper  we  will  consider  only  catadiop¬ 
tric  based  sensors.  Often  such  systems  consist  of  a  camera 
pointing  at  a  convex  mirror,  as  in  figure  (1). 

How  to  interpret  and  make  use  of  the  visual  informa¬ 
tion  obtained  by  such  systems,  e.g.  how  they  should  be 
used  to  control  robots,  is  not  at  all  obvious.  There  are  in¬ 
finitely  many  different  shapes  that  a  mirror  can  have,  and 
at  least  two  different  camera  models  (perspective  and  or¬ 
thographic  projection)  with  which  to  combine  each  mirror. 


Camera 

Figure  1 .  The  generic  setup  of  the  type  of  sen¬ 
sor  that  we  consider  in  this  paper. 


The  properties  of  the  resulting  sensors  are  very  sensitive  to 
these  choices. 

The  classic  need  for  wide  angle  lenses  has,  of  course 
been  in  photography.  In  particular,  underwater  and  archi¬ 
tectural  photography  are  two  examples  in  which  having  a 
wide-angle  lens  is  often  crucial.  The  commercially  avail¬ 
able  lens  with  the  widest  field  of  view  (without  radial  dis¬ 
tortion)  that  the  authors  are  aware  of  is  the  Nikon  13mm 
f/5.6  Nikkor  AIS,  which  provides  a  field  of  view  of  118 
degrees  at  a  cost  of  $(US) 12000.  Note  that  our  prototype 
orthographic  sensor  provides  a  field  of  view  of  142  degrees. 

1  Related  work 

In  the  past  few  years,  there  has  been  a  tremendous  in¬ 
crease  in  research  on  the  design  and  applications  of  cata¬ 
dioptric  based  sensors.  Much  of  this  work  has  been  focused 
on  designing  sensors  with  a  panoramic  or  wide  field  of  view 
(see  [16],  [10],  [13],  [17],  [9],  [7]  [11],[18]  [3],  [19],  [15], 
[12],  [14],  [4],  [8],[1],  [6] ). 

In  [14],  Nayar  describes  a  true  omni-directional  sensor. 
In  this  case,  the  goal  was  to  reconstruct  perspective  views. 
This  sensor  uses  a  parabolic  mirror,  which  is  essentially  the 
only  shape  from  which  one  can  do  a  perspective  unwarping 
of  the  image  when  using  a  camera  that  is  well  modeled  by 
an  orthographic  projection  (see  [1]). 

A  different  use  of  catadioptric  sensors  is  an  application 


of  Mouadddib  and  Pegard  [15].  In  this  case  a  conical  mirror 
is  used  to  estimate  a  robot’s  pose.  This  is  done  using  verti¬ 
cal  lines  in  the  world  as  landmarks,  which  appear  as  radial 
lines  in  the  image.  If  the  positions  of  these  landmarks  are 
known,  then  they  can  be  used  to  estimate  the  robot’s  pose. 
In  contrast  to  [14],  in  this  work  the  authors  use  their  device 
as  a  2D  sensor.  The  effect  of  noise  on  computing  a  robot’s 
position  by  measuring  the  angles  between  known  landmarks 
is  investigated  in  [2].  Navigation  and  map  building  with  a 
mobile  robot  using  a  conical  mirror  is  considered  by  Yagi  et 
al  in  [19]  and  [18]. 

In  [4],  Chahl  and  Srinivasan  describe  a  means  of  esti¬ 
mating  range  by  moving  a  panoramic  sensor,  based  on  the 
fact  that  the  local  distortion  of  the  image  is  range  depen¬ 
dent.  This  method,  which  gives  a  range  estimate  in  every 
azimuthal  direction,  is  implemented  using  a  conical  mirror. 

The  work  most  related  to  this  paper  is  [6]  and  [5].  In 
[6],  Conroy  and  Moore  derive  a  family  of  mirrors  for  which 
the  resolution  in  the  image  is  invariant  to  changes  in  eleva¬ 
tion.  In  [5]  the  authors  exhibit  a  family  of  reflective  surfaces 
that  preserve  a  linear  relationship  between  the  angle  of  inci¬ 
dence  of  light  onto  a  surface  and  the  angle  of  reflection  onto 
the  imaging  device. 

Finally,  we  should  point  out  that  a  preliminary  form  of 
this  work  is  [8].  In  this  work  the  very  same  mirrors  are 
discussed  that  we  consider  here.  There  is  a  major  difference 
though,  namely  that  at  that  point  the  authors  were  unaware 
that  these  mirrors  approximated  perspective  projections  in 
general,  i.e.  we  believed  that  they  only  would  unwarp  the 
single  plane  that  they  were  modeled  on.  It  was  after  this 
work  that  the  authors  noticed  this  property  experimentally 
and  proved  the  approximation  that  appears  below. 


Figure  2.  Here  we  see  a  panoramic  sensor  on 
a  floor  surrounded  by  8.5  inch  square  sheets 
of  paper. 


Figure  3.  On  the  left  we  see  a  checkerboard 
scene  similar  to  the  one  in  figure  (2),  but  now 
from  the  viewpoint  of  a  catadioptric  sensor 
consisting  of  a  spherical  mirror  and  a  stan¬ 
dard  camera  that  gives  an  approximate  per¬ 
spective  projection.  On  the  right  the  view¬ 
point  of  a  sensor  that  uses  a  parabolic  mir¬ 
ror  coupled  with  a  camera  that  gives  an  ap¬ 
proximate  orthographic  projection.  (This  sys¬ 
tem  was  purchased  from  Cyclovision  Inc.) 
In  each  case,  the  mirrors  are  approximately 
thirty  centimeters  above  the  ground.  Notice 
how  the  size  of  the  squares  decreases  as  a 
function  of  their  distance  from  the  camera 
and  that  the  distortion  caused  by  the  spheri¬ 
cal  mirror  is  much  greater  than  the  distortion 
caused  by  the  parabolic  mirror. 


2  Contributions 

In  figure  (2)  we  see  a  scene  consisting  of  a  checkerboard 
pattern  spread  out  on  the  floor  around  a  panoramic  sensor. 
Images  of  this  scene  (and  different,  but  similar  scenes)  taken 
from  sensors  using  spherical  and  parabolic  mirrors  appear 
in  figure  (3).  It  is  clear  that  the  distortion  caused  by  the 
spherical  mirror  is  greater  than  that  caused  by  the  parabolic 
mirror. 

In  this  paper  we  present  a  class  of  sensors  that  provide  a 
wide  field  of  view  with  a  perspective-like  projection  with¬ 
out  any  processing  in  software.  In  particular,  it  is  possible 
to  create  a  mirror  that  does  not  distort  the  checkerboard  at 
all,  other  than  by  a  chosen  scaling  factor.  Images  taken  from 
such  a  sensor  appear  in  figures  (4)  and  (5).  The  key  to  find¬ 
ing  the  shape  of  this  mirror  is  to  write  down  the  relationship 
between  the  equation  of  the  mirror  and  how  it  distorts  the 
checkerboard.  This  equation  will  contain  the  derivative  of 
the  function  describing  a  cross  section  of  the  mirror,  and 
may  be  considered  as  a  means  for  finding  the  distortion  if 
the  mirror  shape  is  given.  On  the  other  hand  it  can  be  con¬ 
sidered  as  a  differential  equation  in  the  shape  of  the  mir¬ 
ror  if  the  distortion  function  is  given.  By  prescribing  the 
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Figure  4.  On  the  right  we  see  a  checkerboard 
scene  (like  that  in  figure  (2),  but  with  fewer 
checkers)  from  the  viewpoint  of  a  sensor  that 
employs  a  new  type  of  mirror.  The  shape 
of  this  mirror  was  determined  by  numerically 
solving  a  non-linear  differential  equation.  In 
this  image,  the  mirror  is  approximately  thirty 
centimeters  above  the  checkerboard  pattern, 
just  as  the  spherical  and  parabolic  mirrors 
were  in  figure  (3).  On  the  left  we  see  the  mir¬ 
ror  that  was  used  to  obtain  this  image. 


Figure  5.  An  image  taken  with  a  catadioptric 
sensor  employing  a  pinhole  mirror. 


distortion  to  be  linear  and  solving  the  differential  equation 
numerically,  data  points  describing  the  cross  section  were 
generated,  which  were  then  used  to  make  prototypes  out 
of  steel  or  aluminum  on  a  CNC  lathe  or  mill.  The  cost  of 
making  each  of  these  prototypes  was  approximately  $  (US) 
700.00. 

3  Prescribing  the  Distortion 

In  this  section  we  derive  the  an  equation  that  leads  to  the 
construction  of  one  of  the  two  different  types  of  mirrors. 
One  model  is  based  on  the  perspective  projection  and  the 
other  based  on  an  orthographic  projection.  For  this  reason 
we  will  refer  to  the  two  mirrors  as  the  “pinhole  mirror”  and 
the  “orthographic  mirror”.  The  pinhole  mirror  is  more  nat¬ 
ural  in  the  sense  that  the  pinhole  camera  is  a  good  model  for 
the  imaging  devices  used  in  many  applications.  On  the  other 
hand,  an  orthographic  projection  is  not  difficult  to  achieve 
using  the  appropriate  optics,  and  the  mathematics  associ¬ 
ated  with  it  is  often  simpler  than  for  the  pinhole  model.  We 
will  omit  the  perspective  model  for  reasons  of  space. 

It  is  clear  from  figure  (3)  that  if  an  object  is  on  the  floor, 
then  the  planar  distance  from  the  optical  axis  of  the  cam¬ 
era  to  any  visible  point  of  the  object  that  touches  the  floor 
is  a  monotonic  function  of  the  pixel  distance  in  the  image. 
Therefore  we  have  a  distance  function  d,  which  takes  pixel 
distances  in  the  image  and  returns  real  world  distances  in 
the  plane.  It  can  also  be  seen  from  figure  (3)  that  the  dis¬ 
tance  function  is  rapidly  increasing  and  approaches  infinity 
as  the  horizon  line  is  approached.  It  seems  natural  then  to 


consider  whether  the  mirror  shape  could  be  altered  so  that 
the  distance  function  would  be  linear  (or  some  other  desired 
function),  as  is  indicated  in  figure  (6).  We  will  refer  to  any 
such  mirror  with  a  linear  distance  function  as  a  rectifying 
mirror. 

We  begin  by  deriving  the  equation  for  d  for  an  arbitrary 
mirror.  Consider  an  orthographic  camera  pointing  up  at  a 
curved  mirror,  as  is  schematically  depicted  in  figure  (7). 
Here  we  see  a  cross  section  of  the  system,  which  is  all 
we  need  consider  since  the  mirror  is  rotationally  symmet¬ 
ric.  Our  goal  is  to  find  an  expression  for  d{x)  given  the 
equation  of  the  cross  section  of  the  mirror,  F>  and  a  point 
whose  distance  from  the  optical  axis  is  x  in  the  image  plane. 

From  the  diagram  we  have  that  tan(0)  =  Ff(x)y  so  that 
tan (20)  =  On  the  other  hand  the  diagram  im¬ 
plies  that  tan(20)  =  Thus  we  have  the  equation 

2 Ff(x)  _  d(x)  —x  . 

1  —  Ff(x)2  F{x)  * 

There  are  two  ways  to  view  equation  (1).  The  first  is 
what  we  just  described  above,  i.e.  if  one  knows  Fy  it  may 
be  substituted  into  the  above  equation  to  determine  d(x), 
which  is  how  the  curves  in  figure  (6)  were  computed.  On 
the  other  hand,  one  could  choose  d(x)  and  then  consider  (1) 
to  be  a  differential  equation  satisfied  by  F.  If  we  solve  this 
differential  equation,  then  the  resulting  mirror  will  have  d  as 
its  distance  function.  It  is  important  to  note  though,  that  at 
this  point  we  know  only  that  this  property  holds  only  in  the 
one  chosen  plane,  and  does  not  other  parallel  planes.  The 
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Figure  6.  Plots  of  the  distance  functions  for  a 
conical  mirror,  a  spherical  mirror,  and  a  recti¬ 
fying  mirrors,  which  we  define  as  any  mirror 
whose  distance  function  is  linear.  Notice  that 
the  distance  function  for  the  conical  mirror 
grows  very  linearly  at  first,  but  does  not  start 
at  0  cm,  i.e.  it  omits  a  region  of  the  plane 
surrounding  the  sensor. 


Figure  8.  The  cross  section  of  the  ortho¬ 
graphic  mirror  used  to  create  the  image  in 
figure  (9). 


reason  for  this  is  that  the  collection  of  rays  that  pass  through 
two  planes  and  reflect  off  of  the  mirror  onto  the  image  plane 
cannot  be  extended  through  the  mirror  to  meet  at  a  single 
point,  i.e.  the  correspondence  between  the  planes  is  not  a 
perspective  mapping.  This  point  is  discussed  in  more  detail 
below.  j 

A  natural  choice  for  the  distance  function  is  d(x)  =  ax 
in  equation  (1).  Considered  as  an  differential  equation,  (1) 
is  non-linear  and  numerical  methods  are  called  for.  For  our 
prototype  orthographic  mirror  we  chose  a  radius  of  1.85 
centimeters,  d(x)  —  54x  and  the  initial  value  F(0)  =  34. 
Hence  the  field  of  view  was  arctan((54  •  1.85)/34))  ~  142 
degrees.  The  resulting  cross  section  can  be  seen  in  figure 
(8). 

In  figure  (9)  we  see  an  image  taken  using  a  mirror  cre¬ 
ated  from  the  cross-section  depicted  in  figure  (8).  (As  is 
evident  from  this  example,  the  prototypes  were  of  low  op¬ 
tical  quality,  and  the  authors  hope  to  have  superior  versions 
of  them  made  in  the  future.)  The  images  presented  earlier, 
in  figures  (4)  and  5)  are  from  a  perspective  mirror. 


4  Approximating  Perspective  Projections 

Our  above  model  was  derived  by  considering  how  the 
sensor  transformed  a  single  plane,  which  we  will  always  re¬ 
fer  to  as  the  floor.  While  for  both  the  pinhole  mirror  and  the 
orthographic  mirror  it  is  clear  from  experiments  and  simu¬ 
lations  that  planes  perpendicular  to  the  optical  axis  will  be 
scaled  by  a  constant,  it  is  possible  to  show  mathematically 
that  with  the  properly  chosen  parameters,  these  mirrors  will 
actually  approximating  a  perspective  projection  to  a  high 
degree.  In  this  section  we  derive  the  approximation  for  the 
orthographic  mirror. 

At  first  it  may  appear  that  a  rectifying  mirrors  should 
only  scale  planes  and  not  distort  them,  but  in  fact  they  do 
both  distort  them  a  little.  In  order  to  see  how  an  arbitrary 
plane,  P,  is  imaged,  we  need  to  know  how  P  is  mapped 
to  the  floor.  If  it  is  to  be  proportionally  imaged,  i.e.  only 
transformed  by  a  scale  factor,  then  it  must  be  transformed 
onto  the  floor  by  a  scaling  factor.  For  this  to  occur,  the 
light  rays  that  are  entering  the  sensor  by  reflecting  off  of  the 
mirror  must  have  the  property  that,  when  extended  beyond 
the  mirror,  they  all  intersect  in  a  common  point  (see  figure 
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Figure  9.  The  image  of  two  collections  of 
checkers  from  two  different  distances.  Here 
the  sensor  was  placed  on  the  edge  of  a  table 
top,  and  one  pattern  of  checkers  was  placed 
around  its  base  on  the  table  while  another  lay 
on  the  floor  next  to  the  table. 


(10)),  i.e.  there  needs  to  be  a  single  “effective  viewpoint” 
for  the  sensor.  Such  a  point  does  not  exist  for  our  two  types 
of  sensors,  because,  as  is  shown  in  [14],  the  only  two  cata- 
dioptric  sensors  with  this  property  are  the  parabola  coupled 
with  an  orthographic  projection  and  the  hyperbola  coupled 
with  a  pinhole  projection.  Finally,  we  know  that  our  sur¬ 
faces  are  not  paraboloids  and  hyperboloids,  because  as  can 
be  checked,  parabolas  and  hyperbolas  are  not  solutions  to 
the  appropriate  differential  equations. 

We  can  demonstrate  why  the  orthographic  mirror  gives 
a  perspective-like  projection  by  computing  exactly  how 
points  in  P  are  scaled  onto  the  floor.  In  figure  (1 1)  we  see 
that  a  point  in  P  with  distance  d!  from  the  optical  axis  is 
mapped  to  a  point  of  distance  d{ x),  which  in  turn  is  mapped 
to  a  point  in  the  image  plane  of  distance  x  from  the  optical 
axis.  We  suppose  that  the  distance  between  the  two  planes 
is  r  and  the  height  of  the  mirror  is  F(0) .  Then  clearly 


F(x) _ r 

d(x)  -  x  ~  d'  -  d(x)  ’ 
from  which  it  follows  that 

...  .  rd(x)  -rx  +  F(x)d(x) 

d(x)  = - W) - 


(2) 

(3) 


Recall  that  d{x)  =  ax  in  our  model,  where  generally  we 
think  of  alpha  as  large  (a  is  54  for  our  prototype).  Hence 


These  lines  do  not  intersect 


Figure  10. 


(r-£  +  F(x))d(x) 
F(x) 


(4) 


Since  we  generally  consider  alpha  to  be  large,  we  have 
the  approximation 


d!{x)  =  T-^-d{x)  (5) 

For  our  prototype  orthographic  mirror,  F{ 0)  was  chosen 
to  be  34  cm,  and  x  varied  from  0  to  1.85  cm.  The  maximum 
value  of  F  occurs  at  x  =  1.85  cm  with  F(1.85)  =  34.85 
cm.  Thus  for  that  mirror  F( 0)  ~  F{x),  which  gives  our 
final  approximation: 


d! 


r  +  F{  0) 

F(0) 


d(x). 


(6) 


This  last  equation  implies  that  this  particular  ortho¬ 
graphic  mirror  will  approximate  a  perspective  projection 
with  a  pinhole  placed  at  (0,  F(0)). 

Finally,  for  the  purpose  of  comparison,  it  would  be  ideal 
if  it  was  possible  to  compare  images  obtained  from  the  or¬ 
thographic  mirror  with  an  image  taken  with  a  lens  providing 
the  same  field  of  view.  As  mentioned  above,  the  widest  pos¬ 
sible  field  that  can  be  obtained  from  a  commercially  avail¬ 
able  lens  is  1 18  degrees.  Thus,  a  simulation  is  a  reasonable 
way  to  perform  such  a  comparison.  For  this  we  used  the 
Persistence  of  Vision  raytracer,  which  will  simulate  a  pin¬ 
hole  camera  with  any  given  field  of  view1 . 

Consider  the  image  in  figure  (12).  In  this  scene  we  see 
a  desk  with  some  familiar  objects.  A  pair  of  spheres  are 
floating  above  the  desk  and  to  the  right  of  the  table  a  por¬ 
tion  of  a  large  box  is  visible.  There  are  a  number  of  other 


lThe  desk  scene  by  Tom  Price  and  Dan  Farmer  and  the  teapot  by 
Alexander  Enzmann  arc  both  POV-Ray  sample  files. 
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Figure  11.  We  consider  a  single  ray  of  light 
passing  through  a  plane  P  and  then  the  floor. 
The  goal  is  to  derive  an  approximation  to 
show  that  planes  other  than  the  floor  map 
onto  the  floor  by  scaling. 


Figure  12.  Here  we  see  a  scene  created  with 
the  POV-Ray  raytracer.  In  addition  to  the  fa¬ 
miliar  objects  on  the  desks  are  many  blue 
spheres,  only  two  of  which  are  visible  in  this 
image.  To  the  right  of  the  desk  can  be  seen 
the  edge  of  a  large  box.  The  camera  has  a 
field  of  view  of  67  degrees. 


blue  spheres  to  the  left  of  the  two  that  are  visible,  but  these 
cannot  be  seen  because  the  field  of  view  is  only  67  degrees. 
This  image  is  formed  by  using  a  perspective  projection. 

In  figure  (13)  we  see  the  same  scene  imaged  with  the 
orthographic  mirror.  The  mirror  has  been  placed  in  the  same 
position  as  the  camera  was  when  figure  (12)  was  created. 
The  table  top  objects  on  the  table  are  relatively  undistorted, 
as  is  the  box,  now  fully  visible  on  the  left.  Since  the  field 
of  view  has  been  increased,  we  can  now  see  all  of  the  blue 
spheres  and  the  entire  box. 

In  figure  (14)  we  see  the  same  desk  scene  again,  this  time 
with  a  simulated  camera  whose  field  of  view  is  142  degrees. 
These  two  images  agree  to  a  great  extent,  with  the  obvi¬ 
ous  exception  of  the  reflection  through  the  Vertical  axis,  and 
demonstrate  the  fineness  of  the  approximation. 

5  Conclusion 

We  have  exhibited  a  sensor  design  which  has  the  ability 
to  give  a  normal  camera  an  ultra- wide  field.  These  sensors 
are  based  on  a  family  of  mirrors  derived  as  numerical  so¬ 
lutions  of  non-linear  differential  equations  which  describe 
how  a  plane  perpendicular  to  the  optical  axis  of  the  system 
is  distorted.  By  using  the  geometry  of  the  mirror,  the  im¬ 
age  is  unwarped  in  an  analog  manner,  and  so  requires  no 
processing  time,  and  thus  these  devices  may  be  considered 
as  “computational  analog  sensors”.  These  sensors  could  be 
useful  for  applications  such  as  human  monitored  surveil¬ 
lance  systems  and  would  not  require  a  digital  computer.  In 


addition,  if  one  did  have  a  computer  available,  applications 
such  a  motion  detection  are  simplified  since  the  sensor  will 
provide  a  uniform  resolution  image  of  the  floor  of  a  room, 
as  opposed  to  a  spherical  or  parabolic  mirror  in  which  the 
outer  regions  would  suffer  from  low  resolution. 
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